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^ ' Abstract 

< ■ 

^ ■ In this paper, we generalize the well known cut-set bound to the problem of lossy transmission of 

' functions of arbitrarily correlated sources over a discrete memoryless multiterminal network. 

HH ■ I. Introduction 

, A general multiterminal network is a model for reliable communication of sets of messages among 

^ I the nodes of a network, and has been extensively used in modeling of wireless systems. It is known 

^ ■ that unlike the point-to-point scenario, in a network scenario the separation of the source and channel 

^ i codings is not necessarily optimal [4]. In this paper we study the limitations of joint source-channel 

■ coding strategies for lossy transmission across multiterminal networks. 
^ ! 

O I A discrete memoryless general multiterminal network (GMN) is characterized by the conditional 

0\ . 

O ' distribution 

■ where X^*) and y(*) (1 < i < m) are respectively the input and the output of the channel at the i*'^ party. 
In a general multiterminal channel with correlated sources, the m nodes are observing i.i.d. repetitions 
of m, possibly correlated, random variables VF^*^ for 1 < i < m. The i*^ party (1 < i < m) has access 
to the i.i.d. repetitions of VF(*\ and wants to reconstruct, within a given distortion, the i.i.d. repetitions 
of a function of all the observations, i.e. f^^\W^^\ W^'^\ VF^™)) for some function /'•*•'(•)■ If this is 
asymptotically possible within a given distortion (see section |II] for a formal definition), we call the source 
(M7(i), VF(2)^...^ V^^M) admissible. In some applications, each party may be interested in recovering 
i.i.d. repetitions of functions of the observations made at different nodes. In this case the function 
f('){W^^\W^^\...,W^"'^) takes the special form of /(^•2)(VF(2)), for 
some functions f^^'^\-). 
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The admissible source region of a general multiterminal network is not known when the sources are 
independent except in certain special cases; less is known when the sources are allowed to be arbitrarily 
correlated. It is known that the source— channel separation theorem in a network scenario breaks down 
[4]. In this paper, we prove a new outer bound on the admissible source region of GMNs. Specializing 
by requiring zero distortion at the receivers, assuming that the functions f'^'\W^'^\W^'^\...,W'^"'^) 
(1 < i < m) have the form of (/(*'i)(iy(^)), /(*'2)(1^(2)), /(i.'^)(M/M)), and that the individual 
messages (VF^-?)) are mutually independent, our result reduces to the well known cut-set bound. 
The results can be carried over to the problem of "lossless transmission" for the following reason: 
requiring the i^^ party to reconstruct the i.i.d. repetitions of f^^\W^^\W^'^\...,W^'^^) with arbitrarily 
small average probability of error is no stronger than requiring the i*'^ party to reconstruct the i.i.d 
repetitions of /«(l^(i),l^(2) _ ^^(m)) ^^^^ ^ vanishing average distortion (for details see section 
HH ). Other extensions of cut-set bound can be found in [2] and [5]. Furthermore some existing works 
show the possibility and benefit of function computation during the communication (see for instance 
[3] [6] [7] [8] [9]). 

A main contribution of this paper is its proof technique which is based on the "potential function 
method" introduced in [10] and [11]. Instead of taking an arbitrary network and proving the desired 
outer bound while keeping the network fixed throughout, we consider a function from the set of all 
m-input/m-output discrete memory less networks to subsets of M'^, where M*^ is the set of all c-tuples of 
non-negative reals. We then identify properties of such a function which would need to be satisfied in 
one step of the communication for it to give rise to an outer bound. The generalized cut-set bound is then 
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proved by a verification argument. Properties that such a function would need to satisfy are identified, 
intuitively speaking, as follows: take an arbitrary code of length say n over a multiterminal network. 
During the simulation of the code, the information of the parties begins from the f*'^ party having the 
i.i.d. repetitions of the random variable W^''^; gradually evolves over time with the usage of the network; 
and eventually after n stages of communication reaches its final state where the parties know enough 
to estimate their objectives within the desired average distortion. The idea is to quantify this gradual 
evolution of information; bound the derivative of the information growth at each stage from above by 
showing that one step of communication can buy us at most a certain amount; and conclude that at the 
final stage, i.e. the n*^ stage, the system can not reach an information state better than n times the outer 
bound on the derivative of information growth. An implementation of this idea requires quantification of 
the information of the m parties at a given stage of the process. To that end, we evaluate the function 
we started with at a virtual channel whose inputs and outputs represent, roughly speaking, the initial and 
the gained knowledge of the parties at the given stage of the communication. See Lemma [T] of section 
Hm and the proof of Theorem [T] of section |IV] for a formal formulation. 

The outline of this paper is as follows. In section |II1 we introduce the basic notations and definitions 
used in this paper. Section |lll] contains the main results of this paper followed by section HVl which gives 
formal proofs for the results. Appendices I] and HIl complete the proof of Theorem [T] from section |llll 

II. Definitions and Notation 

Throughout this paper we assume that each random variable takes values in a finite set. M denotes 
the set of real numbers and denotes the set of non-negative reals. For any natural number k, let 
[k] = {1,2,3, ...,k}. For a set S* C [k], let S'^ denote its compliment, that is [k] — S. The context will 
make the ambient space of S clear. 

We represent a GMN by the conditional distribution 

meaning that the input by the i^^ party is X^*) and the output at the i*'^ party is Y^^\ We assume that 
the z*^ party {1 < i < m) has access to i.i.d. repetitions of W^^\ The message that needs to be delivered 
(in a possibly lossy manner) to the z*^ party is taken to be M^*) = f^^\W^^\W^'^\ W^"^^) for some 
function /(*)(•)■ We assume that for any i G [m], random variables Y^'^\ W^^^ and M^*) take values 
from discrete sets y^'^\ W^*^ and M.^^^ respectively. For any natural number n, let (3^^*^)", 
(y\;{i))ri and (M(^))'^ denote the n-th product sets of y^^, W^'^ and M'^'I We use Y^^ to denote 
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TABLE I 

Notations 



Variable 


Description 


R 


Real numbers. 


R+ 


Non-negative real numbers. 




The set {1, 2, 3, k}. 


771 


Number of nodes of the network. 




The statistical description of a 
multi- terminal network. 




Random variable representing the source observed at the node. 




Random variable to be reconstructed, 
in a possibly lossy way, at the z^'^ node. 




Alnhabet sets of X^^^ V^^^ W*^^^ A/^'^ 


A«(-,-) 


Distortion function used by ttie i*'' party. 




The encoding function used by the i*'' party at the fc*'' stage. 




The decoding function at the i*'' party. 




i_/ClIglll Ui LilC L.(JU.C LlSCU.. 


n(.) 


Down-set (Definition |4}; 
Minkowski sum of two sets (Definition O. 


> 


A vector or a set being greater than or equal the other (Definition |4l(. 




A permissible set of input distributions; 
Given input sources and a multiterminal network, 4' is a set of 
joint distributions on X^^^ x A''^' x A"*^' x ■ • • x A"'™'. 
Inputs to the network have a joint distribution belonging to this set. 



For any i € [m], let the distortion function A*^*) be a function A*^*) : TW^*) x M.^^^ [0, oo) satisfying 
TT^l*)) = for all m-W g TW*-*^. For any natural number n and vectors (?n,^*\ 771,2'', mn^) 
and (m/*-*, m2*-*'', mi*-*) from (Al^*))", let 

fc=i 

Roughly speaking, we require the i.i.d. repetitions of random variable M^*) to be reconstructed, by the 
i*^ party, within the average distortion of D^'^\ 
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Definition 1: Given natural number n, an (n)-code is the following set of mappings: 

For any i e [m] : ■ (W^^^)" — > X^^; 

For any ie[m],ke[n]- {1} : (k'^ : (W^^^)" x [y^^)'''^ x'^^; 

For any i e [m] : : (>V«)" x C^^^)" (A^(^))". 

Intuitively speaking C^*' is the encoding function of the i*'' party at the A;*'* time instance, and is the 
decoding function of the i*^ party. 

Given positive reals e and D^*) (!<?'< m), and a source marginal distribution p{w^^\w^'^\ w^"^^), 
an (n)-code is said to satisfy the average distortion interval D^^^ (for all i € [m]) over the channel 
q{y^^\y^'^\ ...,y^"^^\x^^\x^'^\ if the following "average distortion" condition is satisfied: 

Assume that random variables W^^l^ for i G [m] are n i.i.d. repetition of random variables {W^^'> , W^'^^ W^'^^ ) 
with joint distribution p{w''^\w'^'^\ ...,w^"^^). Random variables X^^ and Y^^^ (k G [n], i G [m]) are 
defined according to the following constraints: 

. (1) (2) (m) (1) (2) (m) (1) (2) (m)^ _ 

P\'^l:n-> ^l:n' "•' '^l:n ' ^l:n' ^l:n' "•' ^l:n ' ^lin' ^lin' "•' Vim > ~ 

fe=l k=l fc=li=l 

and that = Cf''(W"S)> and for any 2 < A; < n, X^'^ = Cfe'^(W^i:i,^i:fc_i)- Random variables X^'^ 
and y^^*^ are representing the input and outputs of the i*^ party at the k*^ time instance and satisfy the 
following Markov chains: 



We then have the following constraint for any i G [m]: 



E 



< L>» + e, 



where = T^f , T^i"^^ 

Definition 2: Given positive reals D('), a source marginal distribution p{w^^^ , w^"^^ w^"^^ ) is called 
an admissible source over the chaimel q{y^^'^ , y^^^ , y^"*^ Ix^^^ , x^^^ , x^"^^ ) if for every positive e and 
sufficiently large n, an (n)-code satisfying the average distortion D^'^\ exists. 

The "independent messages zero distortion capacity region" of the GMN, 
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is a subset of m^-tuples of non-negative numbers R^'^'^^ for i,j € [m] defined as follows: consider the set 
of all sets W(^),W(2),...,>vM, functions f^'\W^^\W^^\...,W^"'y) (I < i < m) having the special 
form of 

(/(M)(vf«),/(^'2)(T^(2)),...,/(^."^)(VfM)), 

the distortion functions A^')(m(*),m'(*)) (for 1 < i < m) being equal to the indicator function l[m^^^ ^ 
m'W], dW being set to be zero for all 1 < i < m and admissible sources p{w^^\w^'^\ ...yW^"^'^) for 
which /'^*'^)(T^(^))'s are mutually independent of each other. The capacity region is then taken to be 
the set of all achievable R^''^^ = H{f(^''\W^^)) (for i,j G [m]) given the above constraints. Intuitively 
speaking, i?^*'^^ is the communication rate from i^^ party to the j^^ party. 

Definition 3: For any natural number c and any two sets of points K and L in W^, let K © L 
refer to their Minkowski sum: K Q) L = {vi + f 2 : vi ^ K,V2 G L}. For any real number r, let 
r X K = {r ■ vi : vi E K}. We also define ^ as the set formed by shrinking K through scaling each 
point of it by a factor ^. Note that in general r x K {n x K) (B (r2 x K) when r = ri + r2 but this 
is true when K is a convex set. 

Definition 4: For any two points v\ and V2 in M^, we say vi>V2i^ and only if each coordinate of 
vi is greater than or equal to the corresponding coordinate of V2. For any two sets of points A and B in 
M^, we say ^ < S if and only if for any point ~a ^ A, there exists a point b E B such that ~a < b . 
For a set A G W^, the down-set Il{A) is defined as: n(^) = {if G : 1? < for some G A}. 

Definition 5: Given a specific network architecture q{y^^\y^'^\ ...,y^"^^\xM\x^'^\ ...,x^"^^), and the 
source marginal distribution p{w^^\w^'^\ it may be possible to find properties that the inputs 

to the multiterminal network throughout the communication satisfy. For instance in an interference 
chaimel or a multiple access chaimel with no output feedback, if the transmitters observe independent 
messages, the random variables representing their information stay independent of each other throughout 
the communication. This is because the transmitters neither interact nor receive any feedback from the 
outputs. Other constraints on the inputs to the network might come from practical requirements such as a 
maximum instantaneous power used up by one or a group of nodes in each stage of the communication. 
Given a multiterminal network q{y^^\y^'^\ ...,y^"^^x^^\ x^'^\ xS"''^) and assuming that X^^^ (i G [m]) 
is the set X^*) is taking value from, let ^' be a set of joint distributions on A'(^) x ^(^^ x X'^^^ x ... x 
for which the following guarantee exists: for any communication protocol, the inputs to the multiterminal 
network at each time stage have a joint distribution belonging to the set ^. Such a set will be called a 
permissible set of input distributions. Some of the results below will be stated in terms of this nebulously 
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defined region ^. To get explicit results, simply replace ^ by the set of all probability distributions on 

III. Statement of the results 

Theorem 1: Given any GMN q{y^^\y^'^\ ...,y^'^^\x^^\x^'^\ ...,x*^™^), a sequence of non-negative real 
numbers D^^^ (i G [m]), an arbitrary admissible source W^^^ (i G [m]), and a permissible set of input 
distributions of the network there exists 

• joint distribution q{x^^^ , x^^^ , x^"*) , z) where size of the alphabet set of Z is 2"^— 1 and furthermore 
q{x^^\x^'^\ ...,x^'^^\z) belongs to * for any value z that the random variable Z might take; 

• joint distribution p{m^^\m^'^\ ...,m^'^\w^^\w^'^\ ...,w^"^^) where the average distortion between 
MW = f^'\W^^\W^^\...,W^"^')) and is less than or equal to i.e. AW(mW,M(^)) < 

such that for any arbitrary T C [m] the following inequality holds: 

I{W^^ lieT ; m(J') : j G T^lW^^^ : j G T^) < /(X^ : i e T ; Y^^^ : j G T^lX^^^ : j G T", Z), 
where Y(-'^\y(-^\...,Y^"'\x(^\x(^\...,X("'^ and Z are jointly distributed according to 

q{y^'\y^''\...,y^"'^x^'\x^''\...,x^"'^)-q{x^'\x^^\...,x^^\z). 
Note that here the following Markov chain holds: 

^-x«,x(2),...,xM-FW,F(2),...,yM. 

Discussion 1: The fact that the expressions on both sides of the above inequality are of the same 
form is suggestive. To any given chaimel q{y^^\y^'^\ ...,y^'^^\x^^\x^'^\ ...jX^"^^) and input distribution 
q{x^^\x^'^\ ...jX^'"^), assign the down-set of a vector in R^"* whose k*^ coordinate is defined as 

where is defined as follows: there are 2^ subsets of [m]; take an arbitrary ordering of these sets and 
take Tfe to be the k*^ subset in that ordering (though not required but for the sake of consistency with 
the notation used in the proof of the theorem assume that T2f._i and are the empty set and the full 
set respectively). Next, to any channel q{y'^'^\y^'^\ ...,ij'^''''^^\x'^'^\x^'^\ ...,x'^'^^) and a set of permissible 
input distributions, we assign a region by taking the convex hull of the union over all permissible input 
distributions, of the region associated to the channel and the varying input distribution. A channel is said 
to be weaker than another chaimel if the region associated to the first chaimel is contained in the region 
associated to the second chaimel. 
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Intuitively speaking, given a communication task one can consider a virtual channel whose inputs and 
outputs represent, roughly speaking, the raw and acceptable information objectives at the m parties. Fur- 
thermore, let the only permissible input distribution for this virtual channel to be one given by the statisti- 
cal description of the raw information of the parties. More specifically, given my p{fh'^^\ ...,m'^'^\w^^^ , w^^^) 
such that A(*)(M(*), mW) < L»W holds, consider the virtual channel p(m(i), m^^)^ \w^^) ^w^'^\ ...,w^'^'^) 

and the input distribution p{w^^\w'^'^\ w^"^'^). The inputs of this virtual channel, i.e. W^^\ W^'^\ 
and its outputs, i.e. M^^\ M^'^\ M^"^\ can be understood as the raw information and acceptable infor- 
mation objectives at the m parties. The region associated to the virtual channel ^(m^^) , m^"^) \w^^\ ...,w^'^^) 
and the input distribution p{w^^\w^'^\ ...,w^"^^) would be the down-set of a vector in R^" whose /c*'^ 
coordinate is defined as 

: i G T ; M^^^ : j £ T^\W^^^ : j G T") . 

Theorem[T]is basically saying that this region associated to this virtual channel and the corresponding input 
distribution should be included inside the region associated to the channel q{y^^^ , y^^^ , y^™^ , x^^) , x^™) ). 
Here the complexity of transmission of functions of correlated messages is effectively translated into the 
performance region of a virtual channel at a given input distribution. This virtual channel at the given 
input distribution must be, in the above mentioned sense, weaker than any physical channel fit for the 
communication problem. 

Corollary 1: Given any GMN q{y^^\y^'^\ jx^^^, x^^), ...jX^"^)), the following region forms an 

outer bound on the independent messages zero distortion capacity region (see Definition |2l) of the network: 

|non-negative i?*^*'-'-* for i,j G [m]: for any arbitrary T C [m] 

q(x^^^ , , , z) such that for any z 
g{a;(l),x(2), g "I- and 

size of the alphabet set of Z is 2"* — 1 

R^''^^ < /(X(^) : i G T ; Y'-^^ : j G T'^jX^-'') : j G Z) 
is satisfied. I , 

where yM ^(i) ^(2), X^™) and Z are jointly distributed according to 

g(y«,2/(2),...,y("^)|x«,x(2),...,x(™)).g(x«,x(2),...,x(™),z). 
Remark 1: This bound is sometimes tight; for instance it is tight for a multiple access channel with 
independent source messages when ^' is taken to be the set of all mutually independent input distributions. 
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Remark 2: This bound reduces to the traditional cut-set bound when ^ is taken to be the set of all 
input distributions, and /(X^*' : i e T ; Y^'^ : i e T^lX^'^ ■.ieT'', Z) is bounded from above by ] 



A. The Main Lemma 

During the simulation of the code, the information of the parties begins from the i*^ party having 
W^^l^ and gradually evolves over time with the usage of the network. At the j*'* stage, the i*^ party has 

(i) (i) th 

^i nM j • represent the information state of the whole system at the j stage by the virtual channel 
piw^iliV^i^j , w^vn) and the input distribution p{w^il^, w^^). In order to quantify 

the information state, we map the information state to a subset of M.'^ (c is a natural number) using a 
function 0(.). A formal definition of (p and the properties we require it to satisfy are as follows: 

Let (j){p{y^^\ y^"^^ \x^^\ ...,x^^^), be a function that takes as input an arbitrary m-input/m-output 
GMN and a subset of probability distributions on the inputs of this network and returns a subset of M[j_ 
where c is a natural number. (p{.) is thus a function from the set of all conditional probability distributions 
defined on finite sets and a corresponding set of input distributions, to subsets of 

Assume that the function (p{.) satisfies the following three properties. The intuitive description of the 
properties is provided after their formal statement. Please see Definitions [3] and |4] for the notations used. 

1) Assume that the conditional distribution p{y^^'^ y' y^'^^ y' y^^^y'^*") jx^-'^), x*^^^ ...,x*^™)) sat- 
isfies the following 

y(2)y'(2)^ ...^ yMy'M | ;r W , . . . , X^"^) ) 

= p(y(l),y{2)^...^y(m)|a.(l)^a.(2)^^_^^^(m)) . 

W, y'M x'^"^)), 

where X is a deterministic function of Y^'^ (i.e. H{X'^'^\Y^'^) = {i e [m])). Random 
variable X'^'-^ (for i G [m]) is assumed to take value from set Take an arbitrary input 

distribution q{xi,X2, ■.■,Xm.)- This input distribution, together with the conditional distribution 
p{y^^\y^'^\ ...,y^"^^x^^\x^'^\ ...,x^"^^), impose a joint distribution q{x'^^\x'^'^\ ...,x'^"^^) on 

'This is valid because /(X**' : i e T ; F*^' : j e T'=\X^J> : j G T^, Z) = H{Y^J^ : j £ T'=\X'^^'> : j £ T", Z)-H{Y^^> : 
j € T'lxW : i e [m],Z) = //(F^^^ : j £ T^IX'^' ; j G T", - H{Y^''> : j G T"|XW : i e [m]) < H{Y^'^ : j G 
T'lX^^' : j G T") - //(y(^") : j G T"|X(') : i G [m]) = ^(^^'^ : i e T ; Y'^'^ : j G T^IX'^) : j G T") . 
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X Then the following constraint needs to be satisfied for any arbitrary set 

^ of joint distributions on X'^^^ x A''^^) x • • • x X'^"^^ that contains q{x'^^\x'^'^\ ...,x'^'^^): 

(1) y' (1) , . . . , y M y' M I , . . . , a;^"*) ) 

,{q{xi,...,Xm)}j ^ 

</.(p(yW, |xW , {q{xi, x„)}) 
© , y' (2) , . . . , y' M |x' , . . . , x' ) , *) . 

2) Assume that 

p(yW,..,y('")|xW,..,x('")) = JJl[y(^) =a:«]. 
Then we require that for any input distribution q{xi,X2, .■.,Xm), the set 

</.(p(yW,...,y(-)|x«,...,x(-)),{g(xi,...,rr^)}) 

contains only the origin in MJ^. 

3) Assume that 

p(y«,...,yM|a;W,...,xM)ni^iP(^i|y«)- 
Then we require that for any input distribution q{xi,X2, ■■■,Xm), 

<^(p(y(^) , yM |x(i) , x(-) ), , x„)}) . 
The first condition is intuitively saying that additional use of the channel 

p(yW,y(2),...,yM|xW,x(2),...,xM) 

can expand by at most 

^(p(y'«,y'(2),...,y'M|^'a),x'(2),...,:,'M),*). 

The second condition is intuitively saying that vanishes if the parties are unable to communicate, 
that is each party receives exactly what it puts at the input of the channel. The third condition is basically 
saying that making a channel weaker at each party can not cause <p{.) expand. 
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Lemma 1: For any function satisfying the above three properties, and for any multiterminal 
network 

distortions D^^^ and arbitrary admissible source VF^*) (i E [m]), positive e and (n)-code satisfying 
the distortion constraints and a permissible set of input distributions, we have (for the definition 
of multiplication of a set by a real number see Definition [3]): 

, . . . , mS!^VS:i > • • • ' w^i^n ) ' {Pi^ii , . . . , w'f^'J ) }) C 
n X Convex y^*^) jx^, rr^™)), ^) }, 

where W^'^j^ {i G [m]) are the messages observed at the nodes; m|?^ {i € [m]) are the reconstructions 
by the parties at the end of the communication satisfying 



E 

for any i G [m]. 



< D» + e, 



IV. Proofs 



Proof: [Proof of Lemma [Tl Let random variables xj^^ and Y^^'^ (/c G [n], i € [m]) respectively 
represent the inputs to the multiterminal network and the outputs at the nodes of the network. We have: 

</'(f'('^SyS-l> «^l:nyifl-l' ■^«lSyS-lkl!i' ^-i^ W^lS)> {P(w^l!n' ^l:n' -> '"^S)}) ® 

(j) {p{w'~lly'~ll_2 , wS^iyS_2 , . . . , ki2 > . • • • . ) . . . • • • > ) }) © 

'^(5(y2i,2/?2i,---,ySl4'2i,4'2i,.^^ 

. . . C 

(/> {piw^^l , , • • • , I w'i!^ ) , i2 ^w'^l,-, ) }) © 
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'?^(5(y2i,y?ii,---,yai4'2i,4'2i,.^^ 
<^(9(yl'\ yi") 4"^), ^) ^ (4) 

n X Convex Hull{(/){q(y^^\y^'^\ 
where in equation [T] we have used property (3); in equation |2] we have used property (1) because 

/ (1) (1) (2) (2) (m) (m), (1) (2) (m)^ 

/'V'^l:ntfl:n-1' "^l:ntfl:n-l' '^l:n yi:n-ll"'l:n' "^l:n' "'l:n ^ P\yn ^---Tyn ; -^n ^--^^n ) 

and furthermore i^(^n^ |W^S^i:2-i) = all « G [m], and that 

,,(2) „,(m)| (1) (2) r'^"')) = nfyW yC^) Ul) (2) (m) X 

The definition of permissible sets implies that the joint distribution p{xn\xn\ ■■■,x^n^^) is in in 
equation |3] we have used property (2). In equation IH we first note that the conditional distributions 

Qivl^^ , y^r^ \x?, x'r^ ) 

fori = 1, 2, n are all the same. We then observe that whenever G (j){q{y^^\ 

for i G [n], their average, ^ Ya=i ^ falls in the convex hull of (p{q{y^^\ l^;*-"^'', x^™)), ■ 

Proof: [Proof of Theorem [H The inequalities always hold for the extreme cases of the set T being 
either empty or [m]. So, it is sufficient to consider only those subsets of [m] that are neither empty nor 
equal to [m]. Take an arbitrary e > and an (n)-code satisfying the average distortion condition D^^^ 
(for all i G [m]) over the channel y^™) Ix*-^-*, x^^^ x^*")). Let random variables X^*^ and 

y^^*^ (A; G [n], z G [m]) respectively represent the inputs to the multiterminal network and the outputs at 
the nodes of the network. Also assume that W^^l^ {i G [m]) are the messages observed at the nodes. Let 
M^!^ (i G [m]) be the reconstructions by the parties at the end of the communication satisfying 



E 



< D^^ + e, 



DRAFT 



April 29, 2009 



13 



for any i € [m]. Lastly, let be a permissible set of input distributions. 

We define a function (/>(.) as follows: for any conditional distribution p{y^^\ y^'^\ y^™-* \x^-^\x^'^\ x^"^^] 
and an arbitrary set ^ of distributions on X^^^ x A'(^) x • • ■X^'^\ let 

<A(p(yW,y('\...,y('")|x«,x(2),...,x(™)),*))= (5) 
U v^(p(y«,y(2),...,y(-)|x«,x(2),...,x(-)Mx«,x(2),...,x(^^ 

p(x(i),x<2),...,x('"))G'^ 

where (^(^(y'^-'^), y^'"^ x*^"'^^ x*^^-*, x*-"^^)) is defined as the down-set 1^ of a vector of size c = 

2^-2 whose /c*^ coordinate equals l{X^^ ■ i e Tk ; F^^^ : j € (rfc)^|X(j) : j G (T^)^) where 
is defined as follows: there are 2™ — 2 subsets of [m] that are neither empty nor equal to [m]. Take an 
arbitrary ordering of these sets and take Tk to be the A;*^ subset in that ordering. 

In appendices II-AI II-BI and II-CI we verify that (/>(.) satisfies the three properties of Lemma [T] for the 
choice of c = 2™ — 2. Lemma \T\ thus implies that (for the definition of multiplication of a set by a real 
number see Definition |3]l: 

,/ .^(1) ^(2) (1) (2) (m)^ r , (1) (2) Mm\ 

/ /-(I) -(2) -(m), (1) (2) (m)^ , (1) (2) (m)^^ ^ 

ra X Convex Hull^(l)(q{y^^\y^^\ y('")|xW,x(2),...,a;('")),^)}. 

According to the Caratheodory theorem, every point inside the convex hull of 

0(g(2/«,y(2),...,yM|^(i),^(2),...,^M),^) 



th 



can be written as a convex combination of c + 1 = 2"^ — 1 points in the set. Corresponding to the i 
point in the convex combination {i G [2™ — 1]) is an input distribution qi{x^^\ x^'^\ x^"^^) such that 
the point lies in 

Let p{x^^\ x^'^\ ...,x^"^\ z) = p{z) ■ Qzix^-^^ x^'^\ x^"^^) where Z is a random variable defined on the 
set {1, 2, 3, 2™ — 1}, taking value i with probability equal to the weight associated to the i^^ point 

^For the definition of a down-set see Definition |4] 
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in the above convex combination. The convex hull of (l){q{y^^\y^'^\ ...,y^"^^\x^^\x^'^\ ...,x^"^^),^) is 
therefore included in (see Definition |3] for the definition of the summation used here): 

U J^p(z)x(^(g(y«,...,y(™)|x«,...,x(-)Mx«,...,x(™)|z)). 

q{x^^^ , , ai^™' , z) such that for any z ^ 
6 * and 

size of the alphabet set of Z is 2'" — 1 

Conversely, the above set only involves convex combination of points in 4){q{y'^^\ y'^™) 1^;^^^ x^"^)), 'I') 
and hence is always contained in the convex hull of (j){q{y^^\ ...,y^"^^ \x^^\ x^"^^), Therefore it must 
be equal to the convex hull region. 
Hence, 

/ /-(I) -(2) -(»n)i (1) (2) (m)^ , (1) (2) (m)^^ ^ 

nx U ^p(z) X(^(g(y«,...,y(-)|x«,...,x(™))g(x«,...,x("^)|z)). 

, x^^^l , x'™^ , z) such that for any z ^ 
g(x(i),a:(2), ...,x(™)|2) G * and 
size of the alphabet set of Z is 2™ — 1 

The set 

/ /-(I) -(2) -(m)| (1) (2) (m)^ . (1) (2) (m)sN 

:ra' ■■■i^l-.n \'^l:n^ ^l:n' "•' ^l:n /^'I'^^lin' ^l:n' '"^l:ra 

is by definition the down-set of a vector of length 2"^ — 2, denoted here by if, whose fe*^ coordinate is 
equal to 

The vector 1/ is greater than or equal to v whose A;*'^ element equals]^ 



n 



for some W^'^ and M(*) (i G [m]) such that the joint distribution of W^'^^ {i G [m]) is the same as that 
of (i G [m]), and that the average distortion between M^*) = _ j^(m)) ^j^^ ^^(i) 

is less than or equal to + ej^ In Appendix JIJ we perturb random variables M(*) (for i G [m]) and 

^This is because for any arbitrary random variables X" ,Y" , Z" such that {X",Y") is n i.i.d. repetition of {X,Y), we 
have: I{X";Z-\Y") = n//(X|y) - > E^=i ^^(^^1^) " ^^(^pln^s) = E^.i ^(^s; ^sl^) = ■ 
/(XG;2'G|GyG) > n ■ I{Xg;Zg\Yg) where G is uniform over {l,2,...,n} and independent of Random 
variables {Xg,Yg) have the same joint distribution as {X,Y). 

''This is because for any arbitrary pair {Y", Z"), the average distortion between Yg and Zg for G uniform over {1, 2, n} 
and independent of is equal to E[A{Yg,Zg)] = E[E[A{Yg, Zg)\G]] = E" i ^E[A(y9, ^'J] = E[A^{Y" , Z")]. 
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define random variables M'(*) (for i € [m]) sucli that for every i € [m], the average distortion between 
M'(*) and M^^^ is less than or equal to D^^^ (rather than D^^^ + e as in the case of M(*)) and furthermore 
for every k 

-.iGTk ; M'U) : j e {nr\W^^^ : j E (Tfc)^) - 0(T(e)) < 

where t(.) is a real- valued function that satisfies the property that T(e) ^ as e ^ 0. 

Hence the vector v is coordinate by coordinate greater than or equal to a vector v' whose k^^ element 
is defined as 

max (n ■ : i G ; M'U) : j G {TkYlW^^^ : j G (TkY) - n ■ 0(r(e)), 0^ . 

The vector v' must lie in 

/ //-(I) -(2) -(m), (1) (2) (m). , (1) (2) {m)xN 

since it is coordinate by coordinate less than or equal to -u . It must therefore also lie in 

nx J ^p(z) X (^(g(yW,...,y('")|x(i),..,a;(™))g(xW,...,x(™)|z)). 

q{x^^\ ...,x^"^\ z) such that for any z ^ 

...,a;('")|2) G -I- and 
size of the alphabet set of Z is 2"^ — 1 

Please note that since ip{.) is the down-set of a non-negative vector, the above Minkowski sum inside 
the union would itself be the down-set of a vectorial The left hand side can be therefore written as union 
over all q{x'^^\x'^'^\ ...,x'^'^\ z) such that q{x'^'^\x^'^\ ...,x'^'^'^\z) G ^ for every z, of the down-set of a 
vector whose /c*^ coordinate equals : i G ; Y^^^ : j G {TkY\X^^'> : j G Since 

the v' falls inside this union, there must exist a particular q{x''^\x^'^\ x*^™) , z) whose corresponding 
vector is coordinate by coordinate greater than or equal to v' . The proof ends by recalling the definition 
of v' and letting e converge zero. ■ 

'This is because for every two non-negative vectors vt and V2, we have A x Ii{vi) © (1 — A) x Tl{v2) = n(Aiyi + (1 — \)v2) 
for any AG [0, 1]. 
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Appendix I 
Completing the proof of Theorem [T] 

A. Checking the first property of Lemma |7] 

Given the definition of </>(.) in equation |5l one needs to verify that: 

<^(p(y«y«,..,y('")y'(™)|x«,x(2),...,x('"))p(x«,x(^^ C 
U (^(p(y'«,y(2),...,y'M[x«,x'(2),...,x'(-)Mx'«,x(2),...,x(™))). 

p(a;'(i),a;'(2),...,a:'("))G<I' 

Take an arbitrary point 1? inside 

<^(p(y«y«,...,y(")y'(-)|x«,x(2),...,x("))p(x«,x(2),...,x(^^ 
We would like to prove that there exists 

and 

such that vi + V2 >~v . 
Since ~v is inside 

the k^^ coordinate of 'v is less than or equal to /(X^ : i S ; yO)y'0) : j g {TkY\X^^^ : j G (rfc)'=) 
where is defined as in the proof of Theorem [1] 
We have: 

/(XW : i G Tfc ; yOOy'O') : j g {nr\X^^'^ : j G (r,)'=) = 
/(X«:iGTfc ; yO') ; ^ (r^nX^^') : j G (Tfcr) + 
/(XW :tGTk ; y'(^') : j G (r,,)lx(j') : j G y(^') : j G (T^)^). 

The second term can be written as: 

: i G Tfc ; Y'^^^ : j G (r,)^!^^^') : j G (rfc)^ Y^^^ : j G (T,)^) < (6) 
: i G Tfc ; Y'^^^ : j G (r,)^|x(-'') : j G (^,)^ yO-)x'(^') : j G (T^)'^) = (7) 

DRAFT April 29, 2009 
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-.ien ; Y'^^^ : j E (rfc)^|x(^')x'(^')y(^') : j g (Tfc)^) + o = 

/(X'W : i G rfc,X(^-)y(^-) : J G (T,)^ ; Y'^^^ : j G (Tfc)^|X'(^-) : j G (T,)^)- 

II^X^j)Y(j) : j G (T,)^ ; Y'^^^ : j G (r,)^|X'(^') : j G (T,)^) = (8) 

/(x'»:iGTfe ; y' W : j g (T^f [X'^^') : j G (T^D - 

: j G (T,)^ ; Y'^^^ : j G {nTlX'^^^ : j G (T,)^) < 

: i G Tfe ; F'^^') : j G (r,)^|X'(^-) : j G (T,)^) 

where in inequality |6] we have used the fact that |y(^))=0 to add : j G {TkY in the 

conditioning pait of the mutual information term. We have also added : i G T^, but this can not 

cause the expression decrease. In the equations [7] and [8] we have used the following Markov chain 

(y'W : i G H) - (X'W : i G [m]) - (yWx» : i G [m]). 

The /c*'^ coordinate of if is thus less than or equal to 

/(xWiiGTfc ; yO-):jG(rfcr|X(^'):jG(rfcr) + 

/(X'W : i G T, ; Y'(^^ : j G (rfc)lX'(^-) : j G (r,)'=). 

Let k^^ coordinate of vi be 

/(xWiiGTfc ; y(^'):jG(r,r|X(^'):jG(rfcr), 

and the k^^ coordinate of V2 be 

/(X'W : i G T, ; y'^^') : j G (Tfc)^|X'(j-) : j G {nY). 



B. Checking the second property of Lemma [7] 
Our choice of (/>(.) implies 

</>(p(y«,...,y(-)Mi),...,x(")),{g(xi,...,:E„)}) =(^(p(y«,...,y^^^ 

Take an arbitrary point 1? inside the above set. The fc*^ coordinate of 1/ is less than or equal to 
: i G Tfc ; y^-?) : j G (rfe)'=|X(-') : j G (T^)^) where is defined as in the proof of Theorem 
[T] Since Y'^^^ = X'^-'^ for j G [m], the /c*'^ coordinate of 1? would be less than or equal to zero. But 1? 
also lies in M^_, hence it has to be equal to the all zero vector. ■ 
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C. Checking the third property of Lemma [7] 

Given the definition of (/>(.) in equation |5l one needs to verify that: 

Take an arbitrary point 1? inside 

, |xW, x(2), .., x^'"))). 

The k^^ coordinate of 'v is less than or equal to l[X^^ : i e ; Z^^) : j £ (Tfc)'=|X(^) : j G {Tkf) 
where is defined as in the proof of Theorem [U The latter vector itself is less than or equal to a vector, 
denoted here by V, whose k^^ coordinate is equal to l{X^'^ :ieTk ; Y^^'^ : j G {TkYlX^^'^ : j G (Tkf) 
because 

m 

p{y^'\y^'\...J'^'>\x^'\x^'\...,x'^'^^)llp{z,\yi), 

1=1 

implying that for every i G [m], I{Z^'^;G^'^\Y^'^) is zero for G*^*^ defined as follows: 

Since the point v' is inside 
we conclude that 

.., x("^)Mx«, C 



DRAFT 



April 29, 2009 



19 



Appendix II 

We will define random variables M'(') (for i G [m]) such that for any i G [m] 

E[Aj(M'«,M«)] < D^^, 

and furthermore 

I{W(^^ :ieTk ; M'O) : j G (rfc)'^|W(^) : j G (T^)'^) - 0(r(e)) < 
: z G Tfc ; MO) : j G (rjk)'^|^^^(^) : j G (Tj^)'^), 

where T(e) ^ as e ^ 0. 

Intuitively speaking, the algorithm for creating M'(*) is to begin with M(*) (i G [m]), and then perturbs 
this set of m random variables in m stages as follows: at the r*'* stage, we perturb the r*'^ random 
variable so that the average distortion constraint is satisfied while making sure that changes in the mutual 
information terms are under control. 

More precisely, let {G^q\g^o\ ...,G^^^) be equal to (M(i),M(2), ...,MM). We define random vari- 
ables {Ci^^ ,Gf\..., G^""^ ) for r G [m] using (g|.^x, in a sequential manner as follows: 
let Gr := C^*!^ for all i G [m], i ^ r. Random variable G^^ is defined below by perturbing G^^l_^ in a 
way that the average distortion between Gr ^ and M^'') is less than or equal to D^''-* while making sure 
that for any G [2"* - 2], 

-A G^^ : j G {Tkr\W^^^ : j G (rfc)^)-/(W?» : z G T^t ; G?2i : j G (Tfc)'=|W?(^) : j G (Tfe)^) 

is of order 0(Tr(e)) where rr(.) is a real- valued function that satisfies the property that Tr{e) — > as 
e — > 0. Once this is done, we can take M'(') = Gm for all i G [m] and let T(e) = ^^^=1 ^ri^)- 
For any arbitrary k G [2"* — 2], as long as r does not belong to {TkY, the expression 

I{W(^ -.ien ; : j G (r,)'^|W(^') : j G (T^)'^)- 

/(w^WiiGTfe ; G^/i,:je{Tkr\w^^^ -.jeinr), 

(r) 

would be zero no matter how Gr is defined. We should therefore consider only the cases where r 
belongs to {Tj^y. In order to define Gr^\ we consider two cases: 

I) Case D^^^ ^ 0: Take a binary random variable Qr independent of all other random variables defined 
in previous stages. Assume that P{Qr = 0) = ^j^+g P{Qr = 1) = JiX+e • ^^"^ equal 
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to G^j\ if Qr = 1, and be equal to M^'') if Q,. = 0. It can be verified that the average distortion 
between g')^^ and M^**) is less than or equal to 

Take an arbitrary k € [2™ — 2] such that r e T^. Since for any five random variables A, B, B', C, D 
where D is independent of iA,B,C) we have I{A;B'\C) - IiA;B\C) < I{A; B'\BCD),^ we 
can write: 

/(VF«:fGrfc ; Gi^^:jG{nr\W^'^:je{nr)- 

We would like to prove that the last term is of order Tr{e) := 0( j^,^',^^ ). Clearly then Tr{e) 
as e — > since D^"^^ is assumed to be non-zero. The last term above is of order ^(/)_^_^ because: 

i{w^'^ -.ien ; Gp'> : j e {T,r\G';^\w^^'> : j e (^,)^ g,,) = 

0-P(Q^ = 1)+ 

: i G r,, ; Gi^^ : j € (Tfc)^[Gj^2ii^^(-') : j G (Tfc)^ = O) • = 0) < 

: i G H) . P(Q, = 0) = Oi^^J. 

2) Case L'(^) = 0: Let the binary random variable Qr be the indicator function l[Ar(G^^^, M^'')) = 0]. 
Let G^r^ be equal to gJ'^i if Qr = 1, and be equal to M(^) if Qr = 0. The average distortion 
between gI^^ and M^^'^ is clearly zero. Since the average distortion between G^.'^i and M^^'^ is 
less than or equal to e, we get that P{Qr = 0) < where 5min is defined as follows: {M^"^^ 
here refers to the set M^^) is taking value from) 

Smin= jnin Ar(i,j)- 

j, j e M'''^ such that 

Take an arbitrary k G [2™ — 2] such that r ^T^. 

i{w^''>:ien ; Gi^^:je{nr\w^'^:je{ny)- 

«This is because E [A, (g1''',MM)] = E[E[A,(G$.''', MM)|Q,]] = P(Q, = 1)E[A,(G<'2i, M^)] < :5FTT; ' P'"' + 
£) = !)('■'. 

'This is because /(A; B I C) > I(A; B'\C) - I{A; B'\BC) > I{A; B'\C) - I(A; B' D\BC) > I{A; B'\C) - I(A; D\BC) - 
I{A-B'\BCD) ^I{A-B'\C) ~ - I{A; B'\BCD) =I{A;B'\C) - I{A- B'\BCD). 
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-.ieTk ; Gi'l, : j G (rfc)^|I4?(^) : j G (T^)^) = 

: i G rfe|G(^')W?(^) : j e (Tfc)^)- 

: i € TfelG^iT^^^^) : j G (T^)^) < 

+ H{W^^ : i G rfc|G(^')W?(^) : j G Q,)- 

H{W^^ : i G Tfc|Gi!^iW?(^) : j G (rfe)^ Q,) < 

H{Qr) + P(Qr = 0) • H{W^^ : i G rfc|G(^)^l?(^) : j G = O) < 

+ P(Q^ = 0) • H{W^^ : i G [m]). 

Let rr(e) := H{Qr) + P{Qr = 0) • H{W^^ : z G [m]). Since P(Qr = 0) is bounded from above 
by that converges to zero as e ^ 0, Tr{e) too would converge to zero as e ^ 0. 

■ 
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